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CLAIMS 

We claim: 



1 . A method for controlling sequencing of data writes from peripheral devices in a 
multiprocessor computer system having a plurality of memory systems, each processor group 
5 being operatively interconnected to each other processor group, the method including the steps 

of: 

queuing a first data write issued by a peripheral device of the system; 
determining whether said first data write is complete; 
° 7 i , tracking the sequence order of the first and second data writes; . . - 

1 0 s^j^ processing said second write data substantially simultaneously with processing of said 

,j ^ first write data, said processing of the write data using one or more of the 



N memory systems; and 

outputting the processed first write data, and then outputting the processed second 



5 

n 
u, 

□ 



write data only upon completion of said first data write. 



15 



^ 2. The method of claim 1 , wherein outputting the processed write data comprises 

□ 

p J outputting the second write data only after receiving all acknowledges of invalidate signals from 

the first and second data writes. 



r \ 

20 3. The method of claim 1 , wherein third write data of a third\date^ write is 

processed substantially simultaneously with the first and second write data. 



4. The method of claim 3, wherein outputting the processed write data comprises 
outputting the second write data only after receiving all acknowledges of invalidate signals from 
the first and second data writes and outputting the third write data only after receiving all 
25 invalidate signals from the first, second and third data writes. 
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5. The method of claim 1 , wherein the memory system provides non-uniform 
memory access between the groups. 

6. The method of claim 1 , wherein said multiprocessor system includes a common 
data cache system utilized for all the processors. 

5 7. The method of claim 1 , wherein said groups are interconnected by at least one 

crossbar. 



yl 



8. The method of claim 6, wherein said groups are interconnected by a tag and 
address crossbar and by a data crossbar. 

9. The method of claim 1 , wherein said groups are interconnected through a 



10 central hardware device. 

03 

Zl 10. A computer system comprising: 

y first and second interconnected groups of one or more processors each; 

O a peripheral device associated with one of the groups and capable of initiating first and 

Pi 

second data writes producing first and second write data, respectively; 
1 5 a queue capable of sequentially ordering the data writes; 

a completion indicator of the first data write, said indicator being responsive to the write 

data; 

a sequencer responsive to the write data and capable of tracking overlapping of the 
write data; 

20 storage for the first and second write data; and 

output for the first and second write data responsive to the sequencer and the 
completion indicator, 
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wherein the storage for the second write data is capable of accepting the second write 
data before completion of the first data write; and 

wherein the output for the second write data is capable of outputting the second write 
data only after completion of the first data write. 

5 11. The system of claim 10, further comprising common data cache system utilized 

for all the processors. 

12. The system of claim 1 0, further comprising one or more crossbars 
interconnecting the groups. 

13. The system of claim 12, wherein the one or more crossbars comprise a tag and 
1 0 address crossbar and a data crossbar. 

14. The system of claim 10, further comprising a central hardware device 
interconnecting the groups. 

1 5. Apparatus for rnaintaining data ordering while substantially simultaneously 
processing data issued from at least one peripheral device which issues data transactions 

1 5 associated with multiple processor systems utilizing at least two processors associated with a 
memory system comprising: 

memory control operatively connected to said processors, said memory system and 
said peripheral device; 

tag tracking control for data read and write information issued by said peripheral 

20 device; 

address crossbar connected between at least two of said memory controls; 
data crossbar connected between at least two of said memory controls; 
transaction sequencer responsive to transactions issued by said peripheral device; 
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transaction completion store for said transactions; 

memory allocator responsive to said transaction completion store; 

data write preventer controlling a second memory system remote from the memory 
control associated with the peripheral device to prevent issuance of a subsequent data write 
5 before a previous data write is completed; and 

transaction output responsive to the transaction sequencer and the transaction 
completion store. 




1 6. Apparatus for maintaining inbound data ordering while substantially 
simultaneously processing data issued from at least one peripheral computer device which issue 

O 1 0 data transactions associated with utilizing non-uniform memory access with a plurality of 

Q 

jg memory systems interconnected with multiple processor systems utilizing at least two 

th 

" j processors associated with said computer memory system comprising: 

peripheral computer device interface means for receiving data writes in the form of 
s transactions from a peripheral computer device, comprising: 

15 transaction completion means for determining whether a given data write 

§ transaction from said peripheral computer device is complete; 

P transaction management means for tracking the completion state and the 

memory location of a given data transaction; 

data organizer means for storage and queuing of each data transaction order 
20 including inbound data queuing means and inbound data handler means wherein said 

inbound data order queuing means tracks the order of each transaction to maintain the 
sequence of each transaction with the order said transaction was issued from the said 
computer peripheral device and said inbound data handler means stores each said 
transaction being tracked by said inbound data queuing means; 
25 wherein at least two write data transactions from each peripheral computer 

device issued sequentially may be processed substantially simultaneously by the system, 
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and each said data transaction is outputted in the same sequence as issued by said 
peripheral computer device. 




17. A system for controlling sequencing of data writes from peripheral computer 
devices in a multiprocessor system utilizing non-uniform memory access with a plurality of 
5 memory systems, each memory system associated with at least one processor group and a 
common data cache system utilized for all the microprocessors, and each processor group is 
operatively interconnected to each other processor group by a tag and address crossbar, and a 
data crossbar, said system providing for overlapping data write processing to begin processing 
of subsequent data writes prior to completion of previous data writes, comprising: 

H« 

O 1 0 a memory control system for each of said processor groups, each said memory control 

P 

«P system being interconnected through said tag and address crossbar, and a data crossbar; 

yi 

vj memory coupled to each memory control system; and 

fi c a plurality of the sub-systems comprising data processors, the plurality of said data 

8 processors storing and forwarding in said memory multiple sequentially issued write data 

hh 1 5 transaction, from a peripheral computer device, and a respective set of transaction identification 

Q 

y3 tags, including one tag for each data transaction stored by said memory; 

each of the plurality of data processors being coupled to the memory control 
system, for sending memory transaction requests to the memory control system; 

the interface for each of the data processors that has a memory for receiving 
20 transaction requests from the memory control system corresponding to memory 

transaction requests by other ones of the data processors; each memory transaction 
request having an associated address value and a order of issue value; 
the memory control system including: 

transaction handling means for activating each memory transaction request when it 
25 meets predefined activation criteria, and for holding each memory transaction request until the 

predefined activation criteria are met; 
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wherein the predefined activation criteria include an transaction ordering conflict 
criterion that is a function of the address value of each transaction and the order which each 
transaction is issued by said peripheral computer device associated with the memory 
transaction request and the address value of activated memory transaction requests; 
5 a transaction management means that stores active transaction status data representing 

memory transaction requests which have been activated by inbound data handling means, the 
active transaction status data including data for each activated transaction representing an 
address value associated with the transaction; the active transaction status data including data 
representing memory transaction requests received from the plurality of data processors; and 
1 0 memory transaction request means for processing the memory transaction request after 

P it has been activated by the transaction activation means; 

Q 

j- the transaction management means including request completion means for comparing 

til 

C : one not-yet-complete memory transaction request with the stored active transaction status data 

*S f or a]} activated memory transaction requests so as to determine whether activation of the each 

09 

e 15 memory transaction request would violate the predefined activation criteria with respect to any 
y, of the system memory transaction requests; 

wherein the transaction management means holds all transaction requests by any of the 
data processors that violate the predefined activation criteria with respect to any memory 
transaction that has already been activated. 

20 18. The system of claim 1 7, wherein said predefined activation criteria is 

determining whether all previously issued write transactions have been completed. 

1 9. Apparatus for maintaining data ordering while substantially simultaneously 
processing data issued from at least one peripheral computer device which issue data 
transactions associated with multiple processor systems utilizing at least two processors 
25 associated with a computer memory system comprising: 
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memory control means operatively connected to each said processors, memory system 
and peripheral computer devices; 

tagging means for tracking data read and write information issued by said peripheral 
computer device; 

5 address crossbar means for interfacing between at least two of said memory control 

means; 

data crossbar means for interfacing between at least two of said memory control 

means; 

means to track the timing sequence of a first transaction issued by any of said peripheral 
1 0 computer devices; 

means to receive and track the sequence of a subsequently issued transaction from said 



1* peripheral computer device; 

\ j means to determine and store information from each peripheral computer device 

^ transaction relative to the state of completion of the data contained therein; 

m 

s 1 5 means for comparing the state of completion of the data contained in said first 

transaction and said subsequently issued transaction and allocating space in said memory 
% system; 

means for preventing the memory system of a memory control remote from the memory 
control means associated with the peripheral computer device issuing the transactions from 
20 issuing a subsequent data write before a previous data write is completed; and 

means for outputting of said first and said subsequent transaction, sequenced in the 
same order as the original order of said first and said subsequent transaction. 
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